Substituting touch gestures for gui or hardware keys to control audio video play

ABSTRACT

A user can toggle between GUI input and touch screen input with the GUI hidden using touch gestures correlated to respective hidden GUI elements and, thus, to respective commands for a TV and/or disk player sending AV data thereto.

This claims priority from U.S. provisional application Ser. No.61/439,279, filed Feb. 3, 2011.

FIELD OF THE INVENTION

The present application relates generally to substituting touch gestureson, e.g., a touch screen of an audio video playback device for an onscreen graphical user interface (GUI) or hardware keys on a remotecontrol (RC).

BACKGROUND OF THE INVENTION

Televisions and other audio video (AV) devices typically are controlledusing hardware keys on the device or an associated remote control (RC)or by using a RC to wirelessly select elements from a graphical userinterface presented on the display of the device. As understood herein,such methods, while effective, can require somewhat cumbersomemanipulation of hardware keys or GUI selector devices.

SUMMARY OF THE INVENTION

A method for controlling an audio video (AV) device includes causing theaudio video device to change channel up, change channel down, anddisplay an electronic program guide (EPG) using respective non-graphicaluser interface (GUI) element selection touch gestures on a display ofthe device.

If desired, the method can include causing a source of AV sending AV tothe AV device to stop, play, and rewind using respective touch gesturesagainst a display of the device. A user can enable and disable touchgesture control of the AV device.

In another aspect, a consumer electronics (CE) device has a processor, atouch screen display controlled by the processor, and a computerreadable storage medium bearing instructions executable by the processorto present a main graphical user interface (GUI) on the touch screendisplay. The main GUI includes first and second main GUI elementsmanipulable by means of a wireless remote control (RC) at least to causerespective first and second control commands to be implemented by theprocessor, with touch commands to control the display being disabled.The processor presents a toggle element on the touch screen display thatis selectable to hide the main first and second GUI elements and toenable touch commands. First and second touch patterns on the touchscreen display are respectively correlated to the first and secondcontrol commands.

In another aspect, a method includes receiving on a touch screen a touchgesture. The touch gesture is characterized by a predeterminedcontinuous motion across a sequence of areas of the surface of the touchscreen. A single command is implemented in an audio video (AV) deviceresponsive to the touch gesture.

The details of the present invention, both as to its structure andoperation, can best be understood in reference to the accompanyingdrawings, in which like reference numerals refer to like parts, and inwhich:

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a non-limiting example system in accordancewith present principles;

FIGS. 2-12 show various screen shots of the display of the CE deviceshown in FIG. 1; and

FIG. 13 is a flow chart of example logic in accordance with presentprinciples.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring initially to the non-limiting example embodiment show in FIG.1, a system 10 includes an audio video (AV) or consumer electronics (CE)device 12 such as a TV including a TV tuner 16 communicating with a TVprocessor 18 accessing a tangible computer readable storage medium 20such as disk-based or solid state storage. The TV 12 can output audio onone or more speakers 22. The TV 12 can receive streaming video from theInternet using a built-in wired or wireless modem 24 communicating withthe processor 12 which may execute a software-implemented browser 26.Video is presented under control of the TV processor 18 on a TV display28 such as but not limited to a high definition TV (HDTV) flat paneldisplay which may be a touch screen display. User commands to theprocessor 18 may be wirelessly received from a remote control (RC) 30using, e.g., rf or infrared. Audio-video display devices other than a TVmay be used, e.g., smart phones, game consoles, personal digitalorganizers, notebook computers and other types of computers, etc.

TV programming from one or more terrestrial TV broadcast sources 32 asreceived by a terrestrial broadcast antenna 34 which communicates withthe TV 12 may be presented on the display 28 and speakers 22. Theterrestrial broadcast programming may conform to digital ATSC standardsand may carry within it a terrestrial broadcast EPG, although theterrestrial broadcast EPG may be received from alternate sources, e.g.,the Internet via Ethernet, or cable communication link, or satellitecommunication link.

TV programming from a cable TV head end 36 may also be received at theTV for presentation of TV signals on the display 28 and speakers 22.When basic cable only is desired, the cable from the wall typicallycarries TV signals in QAM or NTSC format and is plugged directly intothe “F-type connector” 38 on the TV chassis in the U.S., although theconnector used for this purpose in other countries may vary. Incontrast, when the user has an extended cable subscription for instance,the signals from the head end 36 are typically sent through a STB 40which may be separate from or integrated within the TV chassis but inany case which sends HDMI baseband signals to the TV.

Similarly, HDMI baseband signals transmitted from a satellite source 42of TV broadcast signals received by an integrated receiver/decoder (IRD)44 associated with a home satellite dish may be input to the TV 12 forpresentation on the display 28 and speakers 22. Also, streaming videomay be received from the Internet 46 for presentation on the display 28and speakers 22. The streaming video may be received at the computermodem 24 or it may be received at an in-home modem 48 that is externalto the TV 12 and conveyed to the TV 12 over a wired or wireless Ethernetlink and received at an RJ45 or 802.11x antenna on the TV chassis.

The viewer can control the TV 12 and sources of content therefor such asthe above-described STB 40, IRD 44 (which typically include digitalvideo recorders), a disk player 49, and the like using the RC 30.Assuming a source of video has certain trick play capability, once theTV begins a vide reception session, e.g., by receiving HDMI video from asource, images of trick play buttons on the RC 30 can be presented onthe display 28 as shown in FIG. 2 so that the user can select one of theimages using the RC 30's point and click capability, or by manuallytouching the image on the touch screen display 28. Accordingly, theimages presented on the touch screen display 28 can be thought of as asoft RC.

In the example shown, a channel up/down button image 50 can be selectedto cause the TV 12 to tune up and down channels. A stop button image 52can be selected to cause the source of video to stop playing the video.An on/off button image 54 can be selected to turn the TV on and off. Afast rewind button image 56 can be selected to cause the source of videoto reverse video playback onscreen at a fast speed. A help button image58 can be selected to cause a help menu to appear. A slow rewind buttonimage 60 can be selected to cause the source of video to reverse videoplayback onscreen at a slow speed. A guide button image 62 can beselected to cause an electronic program guide (EPG) to be presented onthe display 28. A pause button image 64 can be selected to pause videopresentation on the display. A play button image 66 can be selected toplay video presentation on the display at normal speed, typically thirtyor thirty two frames per second. A menu button image 68 can be selectedto cause a menu option guide to be presented on the display 28. Aninformation button image 70 can be selected to cause textual informationabout a current video or TV program to be presented on the display 28. Afast forward button image 72 can be selected to cause the source ofvideo to play video onscreen at a fast speed that is faster than normalspeed. A fast forward button image 74 can be selected to cause thesource of video to play video onscreen at a speed that is faster thanthe fast speed. An advance button image 76 can be present to cause videoto be advanced by a chapter or scene. Additional button images may beprovided and are described further below.

As set forth further below, in addition to using the soft keys shown onthe display 28 in FIG. 2, a viewer can input control commandscorresponding to those discussed above using touch gestures. Specificmodes for entering touch gesture input are discussed below. In oneexample, the following gestures on the touch screen display of the TVcan control TV and video playback may be as follows. “Flick” and “swipe”are used interchangeably to mean a touch that is movement of a hand,finger, or stylus against the touch screen generally along a line:

Channel UP

-   -   2 SWIPE up

Channel DOWN

-   -   2 SWIPE down

Guide

-   -   2 LONG PRESS (on release)

Menu

-   -   LONG PRESS (on release)        Play/Pause (for, e.g., controlling the disk player or other        input)    -   DOUBLE TAP        4 way (courser)    -   FLICK/SWIPE up/down/left/right

Select

-   -   TAP

FF

-   -   FLICK/SWIPE right

REC

-   -   FLICK/SWIPE left

REC

-   -   SPIRAL

Next

-   -   2 SWIPE up

Previous

-   -   2 SWIPE down

Stop

-   -   CIRCLE

In other words, a main graphical user interface (GUI) can be shown onthe touch screen display as illustrated in FIG. 2, with the main GUIincluding first and second main GUI elements manipulable by means of awireless remote control (RC) at least to cause respective first andsecond control commands to be implemented by the processor, with touchcommands to control the display being disabled. At least one toggleelement can be shown on the touch screen display and is selectable tohide the main first and second GUI elements and enable touch commands,with first and second touch patterns on the touch screen display beingrespectively correlated to the first and second control commands. Thefirst touch pattern can be two successive swipes up the touch screendisplay starting nearer to a bottom edge of the display and endingnearer to a top edge of the display, and the first control command canbe channel up. Or the first touch pattern can be two successive swipesdown the touch screen display and the first control command can bechannel down. Or the first touch pattern can be two successive longpresses on the touch screen display and the first control command can be“display electronic program guide (EPG)”, with a long press beingcontinuous contact against the display for longer than a predeterminedperiod substantially in one place on the display and with two successivelong presses being two long presses separated from each other temporallyby a time period, causing an EPG to be presented on the display. Yetagain, the first touch pattern can be a single press against the touchscreen display and the first control command can be “show menu”, causinga menu to be presented on the display.

The TV may receive audio video data from a player and the processor ofthe TV can send the following commands to the player when the followingtouch gestures are sensed and touch control is enabled:

(a) Play/Pause (for, e.g., controlling the disk player or otherinput)—DOUBLE TAP toggles between play and pause(b) 4 way (cursor)—FLICK/SWIPE up/down/left/right

(c) Select—TAP

(d) fast forward—FLICK/SWIPE right(e) rewind—FLICK/SWIPE left(f) record—SPIRAL(g) Next chapter—2 SWIPE up(h) Previous chapter—2 SWIPE down

(i) Stop—CIRCLE

The above touch commands can replace the need to have on screen GUI or ahardware remote to control universal TV and Blu-Ray functions.

Note that more granularity for fast forward and reverse while “pause” isselected may be provided for with the following gestures:

With video seeking available:

-   -   Slow Mode Forward (available while paused)    -   FLICK right at flick speed one initiates forward speed one    -   FLICK left at any flick speed returns to pause    -   Second left flick at slow speed initiates forward slow speed two        (different than forward slow speed one)    -   FLICK right at flick speed two initiates forward slow speed two    -   FLICK left at any flick speed returns to forward speed two    -   While in forward slow at any speed, a double tap returns to play        at normal speed    -   While in forward slow at any speed, a tap returns to pause    -   Slow mode rewind (available while paused)    -   FLICK left at flick speed one initiates rewind at slow speed one    -   While in slow speed one, FLICK right at any flick speed returns        to pause    -   Second left flick at any flick speed initiates rewind at slow        speed two    -   FLICK left at flick speed two initiates rewind at slow speed two    -   FLICK right at any flick speed returns to rewind at slow speed        one    -   Second left flick at any speed—no response    -   While in forward slow at any speed, double tap to return to play    -   While in forward slow at any speed, tap to return to pause

In addition to slow motion forward at playback speeds slower than normalplayback speed, fast forward (FF) at faster than normal playback speedcan be initiated as follows:

-   -   FLICK right at flick speed 1 initiate FF speed 1    -   FLICK left at any flick speed return to normal speed    -   Second left flick at speed 1 initiate FF speed 2 (different than        FF speed 1)    -   FLICK right at flick speed 2 initiate FF speed 2    -   FLICK left at any flick speed return to FF speed 1    -   Second right flick at flick speed for 2 initiate FF speed 2    -   FLICK right at flick speed 2 initiate FF speed 3    -   FLICK left at any flick speed return to FF speed 2    -   While in FF at any speed, double tap to return to normal speed    -   Rewind (REW)    -   FLICK left at flick speed 1 initiate REW speed 1    -   FLICK right at any flick speed return to normal speed    -   Second left flick at speed 1 initiate REW speed 2    -   FLICK left at flick speed 2 initiate REW speed 2    -   FLICK right at any flick speed return to REW speed 1    -   Second left flick at flick speed for 2 initiate REW speed 2    -   FLICK left at flick speed 2 initiate REW speed 3    -   FLICK right at any flick speed return to REW speed 2    -   Second left flick at any speed—no response    -   While in REW at any speed, double tap to return to normal speed        With video seeking unavailable:        Slow Mode Forward (available while paused)    -   FLICK right at flick speed 1 initiate forward slow speed 1        -   FLICK left at any flick speed to return to stop playhead        -   Second left flick at any speed no response    -   While in forward slow at any speed, double tap to return to play    -   While in forward slow at any speed, tap to stop the playhaed    -   Slow mode Rewind (available while paused)    -   FLICK left at flick speed 1 initiate REW slow speed 1        -   FLICK right at any flick speed return to stop playhead        -   Second left flick at any speed no response    -   While in forward slow at any speed, double tap to return to play    -   While in forward slow at any speed, tap to stop the playhead

FF

-   -   FLICK right at flick speed 1 initiate move playhead forward        speed 1        -   FLICK left at any flick speed stop playhead        -   Second left flick at speed 1 move playhead forward speed 2    -   FLICK right at flick speed 2 move playhead forward speed 2        -   FLICK left at any flick speed stop playhead        -   Second right flick at flick speed for 2 initiate FF speed 2    -   FLICK right at flick speed 2 move playhead forward speed 3        -   FLICK left at any flick speed stop playhead    -   While in FF at any speed, double tap to return to normal speed    -   End of file, stop playhead and jump back 10 seconds

REW

-   -   FLICK left at flick speed 1 initiate move playhead forward speed        1        -   FLICK right at any flick speed stop playhead.        -   Second right flick at speed 1 move playhead forward speed 2    -   FLICK left at flick speed 2 move playhead forward speed 2        -   FLICK right at any flick speed stop playhead        -   Second left flick at flick speed for 2 initiate FF speed 2    -   FLICK left at flick speed 2 move playhead forward speed 3        -   FLICK right at any flick speed stop playhead    -   While in FF at any speed, double tap to return to normal speed        End of file, stop playhead

As stated above, swipes or flicks generally are touch patterns that moreor less resemble straight lines of predetermined lengths that a usertraces on the display surface without lifting his or her finger (orstylus, it being understood that the other gestures can also be madewith a stylus). Circles generally are touch patterns that generallyresemble a circle that a user traces without lifting his or her fingerfrom the display surface, a long press is a touch pattern in one placeon the display of greater than a predetermined period, a tap is a singletouch on a single area of the display of less than a predeterminedperiod and, if desired, longer than a threshold period, etc. A spiraltouch pattern generally resembles a continuous spiral traced by the userwithout lifting his or her finger from the display surface.

A particular gesture may be sensed by the processor by comparing thetactile pattern sensed by the touch screen display to a library of touchtemplates. If a touch pattern more closely resembles a circle templatethan a spiral template, for example, then the processor returns“spiral”. In general, the processor returns the template (andconcomitant control command from the library above) that the touchgesture sensed by the touch screen most closely matches. When thecommand is for a device (such as the disk player) sending AV to the TVthe TV processor passes the command to the device.

By touching an “icon” element 200 on an onscreen GUI (FIG. 2) or bymaking any touch on the screen, the user can signal to the processorthat touch commands are now to be invoked, and the main GUI 201 can behidden from view except for the three toggle icon display 202 of FIG. 3.By touching a “full” icon 204 (FIG. 3) the entire GUI can be made toreappear as shown in FIG. 2, with touch commands disabled andGUI-initiated commands invoked. By touching a direction pad icon 206,only the three toggle icon display 202 and a four-way direction pad 208(FIG. 4) can be made to appear, with the main GUT shown in FIG. 2removed and with touch gesture control enabled. The arrows on the pad208 can be touched to move a screen cursor in the directions indicatedby the arrows.

Recognizing that several different touches are used to execute variousfunctions that make up the TV and disk player user experience(s),present principles provide for training the user on which touchesaccomplish which features. In one example, on screen animations of ahuman hand making the various touch gestures are overlaid onto the videobeing presented on the display that depict respective gestures inrelation to their respective function. Thus, the animated hand (FIG. 5)moves (as shown in temporal sequence in FIG. 6) on screen through thecorresponding gesture, e.g., through a linear swipe or circle or spiralmotion. In FIG. 6 and using the circle gesture as an example, theanimation begins at 300 and then moves in continuous temporal sequenceas indicated by the arrow 301 through 302, 304, 306, and 308 to form acircular trace, it being understood that only a single hand image isshown at a time and that the positions 300-308 are temporally spacedimages of the same animated hand.

The animations may appear when a corresponding GUI button or RC functionhas been invoked. For example, while watching a video on a touchinterface if there is a GUI button that executes “rewind”, when REW ispressed either using the GUI or a hardware key on the RC an animationappears (in, e.g., the upper right corner of the display 28) depictingthe appropriate gesture for REW. Hand animations illustrating each ofthe above-discussed gestures can be presented on screen when therespective soft key for the same function is selected.

The user may be given the option of turning animations on and off bytoggling a GUI button that has on and off states. Voice commands mayalso be implemented for inputting the above-discussed commands.

FIG. 7 shows that a soft number pad 310 can be presented on the display28 to allow a viewer to select any number from zero to nine to inputfor, e.g., tuning to a specific channel number.

FIGS. 8-10 show a sequence of screen shots that can be presented on thedisplay 28. From a normal video presentation in FIG. 8, a viewer cantouch the display 28 with four fingers, or select a “search” button onthe RC 30 or a corresponding “search” soft key presented on the display28, to invoke the screen shot of FIG. 9. In FIG. 9, a soft keyboard 312which in the embodiment shown is a full QWERTY keyboard can be presentedand its keys selected by touch or by using the RC 30 to input a keywordinto the text box 314 as shown in FIG. 10. An Internet search for thekeyword may then be executed by the TV 12, or a search of the EPG forthe keyword may be executed, or other source may be searched for thekeyword. Results are presented on the display 28.

FIGS. 11 and 12 illustrate that while in full screen video display mode(FIG. 11), a viewer can “drag” the edge of the display by, e.g., placingmultiple fingers on the display and moving his or her fingers toward theedge of the display to cause the split screen shown in FIG. 12 toappear. In the split screen, the video formerly presented in the fullscreen mode of FIG. 11 is presented in a video window 316 in FIG. 12. Adescription window 318 may be juxtaposed with the video window 316 andmay present informational text pertaining to the video in the window 316as derived from metadata received with the video. A browser window 320may also appear with the windows 316, 318 and can presentInternet-sourced information pertaining to the video in the video window316.

FIG. 13 shows example overall logic in accordance with presentprinciples. Commencing at decision diamond 320, if the TV processordetermines that touch gesture control is not enabled, the logic flows toblock 322 to receive input commands from the RC 30 and/or from the softbutton GUI shown in FIG. 2. On the other hand, when touch gesturecontrol is enabled, the logic flows to block 324 to receive, via thetouch screen display 28, one of the above-described tactile gestures,determining which gesture has been received at block 326 using, e.g.,the above-described template method. Other methods may be used withoutlimitation. At block 328 the TV correlates the sensed gesture with thecorresponding command and implements the command on the TV (for, e.g.,channel up/down) or the source of video (for, e.g., pause, play, fastforward, etc.) In the latter case, when the TV processor determines thegesture corresponds to a command that requires execution by a componentcommunicating with the TV, the TV processor sends the command to thecomponent for execution.

While the particular SUBSTITUTING TOUCH GESTURES FOR GUI OR HARDWAREKEYS TO CONTROL AUDIO VIDEO PLAY is herein shown and described indetail, it is to be understood that the subject matter which isencompassed by the present invention is limited only by the claims.

1. Method for controlling an audio video (AV) device comprising: causingthe audio video device to change channel up, change channel down,display an electronic program guide (EPG) using respective non-graphicaluser interface (GUI) element selection touch gestures on a display ofthe device.
 2. The method of claim 1, further comprising causing asource of AV sending AV to the AV device to stop, play, and rewind usingrespective touch gestures against a display of the device.
 3. The methodof claim 1, comprising enabling a user to enable and disable touchgesture control of the AV device.
 4. Consumer electronics (CE) devicecomprising: processor; touch screen display controlled by the processor;and computer readable storage medium hearing instructions executable bythe processor to: present a main graphical user interface (GUI) on thetouch screen display, the main GUI including first and second main GUIelements manipulable by means of a wireless remote control (RC) at leastto cause respective first and second control commands to be implementedby the processor, touch commands to control the display being disabled;present at least one toggle element on the touch screen displayselectable to hide the main first and second GUI elements and enabletouch commands, first and second touch patterns on the touch screendisplay being respectively correlated to the first and second controlcommands.
 5. The CE device of claim 4, wherein the first touch patternis two successive swipes up the touch screen display starting nearer toa bottom edge of the display and ending nearer to a top edge of thedisplay, and the first control command is channel up.
 6. The CE deviceof claim 4, wherein the first touch pattern is two successive swipesdown the touch screen display and the first control command is channeldown.
 7. The CE device of claim 4, wherein the first touch pattern istwo successive long presses on the touch screen display and the firstcontrol command is “display electronic program guide (EPG)”, a longpress being continuous contact against the display for longer than apredetermined period substantially in one place on the display, twosuccessive long presses being two long presses separated from each othertemporally by a time period, causing an EPG to be presented on thedisplay.
 8. The CE device of claim 4, wherein the first touch pattern isa single press against the touch screen display and the first controlcommand is “show menu”, causing a menu to be presented on the display.9. The CE device of claim 4, wherein the CE device receives audio videodata from a video source and the processor sends the following commandsto the video source when the following touch gestures are sensed andtouch control is enabled: (a) Play/Pause—DOUBLE TAP (b) 4 way(cursor)—FLICK/SWIPE up/down/left/right (c) Select—TAP (d) fastforward—FLICK/SWIPE right (e) rewind—FLICK/SWIPE left (f) record—SPIRAL(g) Next chapter—2 SWIPE up (h) Previous chapter—2 SWIPE down (i)Stop—CIRCLE
 10. The CE device of claim 4, wherein the CE device receivesaudio video data from a video source and the processor sends thefollowing command to the video source when the following touch gestureis sensed and touch control is enabled: Play/Pause—DOUBLE TAP.
 11. TheCE device of claim 4, wherein the CE device receives audio video datafrom a video source and the processor sends the following command to thevideo source when the following touch gesture is sensed and touchcontrol is enabled: 4 way (cursor)—FLICK/SWIPE up/down/left/right. 12.The CE device of claim 4, wherein the CE device receives audio videodata from a video source and the processor sends the following commandto the video source when the following touch gesture is sensed and touchcontrol is enabled: Select—TAP.
 13. The CE device of claim 4, whereinthe CE device receives audio video data from a video source and theprocessor sends the following command to the video source when thefollowing touch gesture is sensed and touch control is enabled: fastforward—FLICK/SWIPE right.
 14. The CE device of claim 4, wherein the CEdevice receives audio video data from a video source and the processorsends the following command to the video source when the following touchgesture is sensed and touch control is enabled: rewind—FLICK/SWIPE left.15. The CE device of claim 4, wherein the CE device receives audio videodata from a video source and the processor sends the following commandto the video source when the following touch gesture is sensed and touchcontrol is enabled: record—SPIRAL.
 16. The CE device of claim 4, whereinthe CE device receives audio video data from a video source and theprocessor sends the following command to the video source when thefollowing touch gesture is sensed and touch control is enabled: Nextchapter—2 SWIPE up.
 17. The CE device of claim 4, wherein the CE devicereceives audio video data from a video source and the processor sendsthe following command to the video source when the following touchgesture is sensed and touch control is enabled: Previous chapter—2 SWIPEdown.
 18. The CE device of claim 4, wherein the CE device receives audiovideo data from a video source and the processor sends the followingcommand to the video source when the following touch gesture is sensedand touch control is enabled: Stop—CIRCLE.
 19. Method comprising:receiving on a touch screen a touch gesture, the touch gesture beingcharacterized by a predetermined continuous motion across a sequence ofareas of the surface of the touch screen; and implementing a singlecommand in an audio video (AV) device responsive to the touch gesture.20. The method of claim 19, comprising enabling a user to enable anddisable touch gesture control of the AV device.